• [ENH] For local: use subquery for FTS, unions for int & float metadata expr, is true -> is not null #4556 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

• Can you think of any use case in which the code does not behave as intended? Have they been tested?
• Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
• If appropriate, are there adequate property based tests?
• If appropriate, are there adequate unit tests?
• Should any logging, debugging, tracing information be added or removed?
• Are error messages user-friendly?
• Have all documentation changes needed been made?
• Have all non-obvious changes been commented?

System Compatibility

• Are there any potential impacts on other parts of the system or backward compatibility?
• Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

• Is this code of a unexpectedly high quality (Readability, Modularity, Intuitiveness)

Local-mode query engine: switch to sub-queries (FTS) + union strategy for mixed int/float metadata

Re-implements how local SQLite queries are generated to improve performance and index usage. Full-text search now uses an IN (subquery) pattern instead of outer joins, and numeric metadata comparisons build two independent indexed selects (int + float) merged via UNION DISTINCT. Additionally, boolean key checks previously using is true have been converted to is not null. Extensive regression tests for null-key, FTS/metadata combinations, and edge cases were added.

Key Changes:
• Added create_union_subquery_for_int_float_ops to build UNION of int_value and float_value paths for all primitive operators.
• DocumentExpression now generates IN subquery against EmbeddingFulltextSearch instead of LIKE column join.
• MetadataExpression: removed inline int↔float OR logic; now delegates to the new union helper and ensures key column IS NOT NULL.
• Removed metadata/FTS left-joins in filter_limit_query; relies on subqueries and adds DISTINCT.
• Updated comparisons to use is_not_null() in several places for index friendliness.
• Purged redundant cast logic; large batch of new async tokio tests covering null metadata keys, mixed operator cases, and FTS×metadata interactions.

Affected Areas:
• rust/segment/src/sqlite_metadata.rs (query generation, tests)

This summary was automatically generated by @propel-code-bot